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DETAILED ACTION 

1 . Claims 1-14 are pending for this application. 

Claim Rejections - 35 USC § 103 

2. The following is a quotation of 35 U.S.C. 1 03(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

3. Claims 1, 9, 10, 11, 13 and 14 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Ramamoorthy (U.S Patent Publication No. 20020085219), in view of 
Fitzgibbon et al. ("Multibody Structure and Motion: 3-D Reconstruction of Independently 
Moving Objects," Lecture Notes in Computer Science, Published by Springer Berlin, 
Volume 1842/2000, Pages 891-906, herein after "Fitzgibbon"). 

4. As to claim 1 , Ramamoorthy discloses "Method of providing contour information 
related to images, comprising the steps of: obtaining a set of interrelated images 
(l.sub.1, I. sub. 2, I. sub. 3), (Ramamoorthy, [0017], lines 1-3, where "image data sets 
associated with an object" is read as "interrelated images" because each set is 
read as an image) , (step 26),segmenting said images, (Ramamoorthy,[0059], lines 
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23-26, where removal of the background is read as segmenting the images), (step 
28), extracting at least two contours (10, 12, 14) from the segmentation, 
(Ramamoorthy, [0129], lines 3-5 where "contours of each positive pixel group" is 
read as "at least two contours" further this process occurs in element 182 for 
figure 7 which is after the background removal, or segmentation in element 178), 
(step 30) selecting interest points (J.sub.1-J.sub.12) on at least some of the contours, 
(Ramamoorthy, [0133], lines 2-4, where contour points are read as interest 
points), (step 32), associating, for said extracted contours, interest points (J) with 
corresponding reconstructed points by means of three-dimensional reconstruction, 
(Ramamoorthy, [0156], lines 1-3, where transformed contours is read as extracted 
contours and the contour points are read as the interest points), (step 34), such 
that at least a reasonable part of a contour of an object can be determined based on the 
linked points (Ramamoorthy,[0156], lines 1-3 where "a three-dimensional surface 
model estimate" which is derived from "transformed contours" is read as "a 
reasonable part of a contour of an object" that the set of N points [0150] from the 
"transformed contours" is read as the "linked points")." 

Ramamoorthy does not specifically disclose "projecting the reconstructed points 
(P.sub.1-P.sub.12) into each image, (step 36), and linking, for each image, 
reconstructed points that are not projected at a junction point between different contours 
or their projections to each other in order to provide a first set of links, (step 38)." 
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Fitzgibbon, however, specifically discloses "projecting the reconstructed points (P.sub.1- 
P.sub.12) into each image, (Fitzgibbon, section 3.2, paragraph 1, lines 3-5, where 
"3D point Xp" is read as the reconstructed point) (step 36), and linking, for each 
image, reconstructed points that are not projected at a junction point between different 
contours or their projections to each other in order to provide a first set of links, (step 
38), (Fitzgibbon, section 3.1, paragraph 1, lines 8-10, where "3D point" is read as 
the reconstructed point, tracks are read as "linking", and the interface between 
the background and foreground objects are read as "a junction point between 
different contours")." 

It would have been obvious to one of ordinary skill in the art at the time of the invention 
to use the teachings of Fitzgibbon with the teachings of Ramamoorthy for improved 3-D 
object reconstruction because of the "advantages of multibody 3D reconstruction" 
(Fitzgibbon, Section 1, paragraph 6, line 1). 

5. As to claim 9, the combination of Ramamoorthy and Fitzgibbon disclose "wherein 
the reconstructed points are provided in a three dimensional space." (Fitzgibbon, 
section 3.1, paragraph 1, lines 8-10, where "3D point" is read as the reconstructed 
point in a three dimensional space. This is also found in Ramamoorthy [0156], 
lines 1-4). 
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6. As to claim 10, the combination of Ramamoorthy and Fitzgibbon disclose, 
"wherein the images are provided in a two dimensional space." (Ramamoorthy, [0040], 
lines 7-10, where X by Y images is read as two dimensional space). 

7. As to claim 1 1 , the combination of Ramamoorthy and Fitzgibbon disclose, 
"further comprising the step of determining the actual motion of contours from image to 
image before projecting reconstructed points into an image." (Fitzgibbon, section 3.2, 
paragraph 1, lines 3-11, "where determining the actual motion of contours from 
image to image" is accomplished by the minimization in equation 9, and "3D 
point Xp" is read as the reconstructed point.). 

8. As to claim 13 Ramamoorthy discloses "an apparatus (16) for providing contour 
information related to images, comprising: an image obtaining unit (18) arranged to 
obtain a set of interrelated images, (Ramamoorthy,[0017], lines 1-3; where "image 
data sets associated with an object" is read as "interrelated images") and an 
image segmenting unit (20) arranged to segment said images, (Ramamoorthy,[0059], 
lines 23-26, where removal of the background is read as segmenting the images), 
and a contour determining unit (22) arranged to: extract at least two contours from the 
segmentation made by the segmentation unit, (Ramamoorthy, [0129], lines 3-5 where 
"contours of each positive pixel group" is read as "at least two contours" further 
this process occurs in element 182 for figure 7 which is after the background 
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removal, or segmentation in element 178), select interest points on the contours of 
each image, (Ramamoorthy, [0133], lines 2-4, where contour points are read as 
interest points), associate, for each extracted contour, interest points with 
corresponding reconstructed points by means of three-dimensional reconstruction, 
(Ramamoorthy, [0156], lines 1-3, where transformed contours is read as extracted 
contours and the contour points are read as the interest points), such that at least 
a reasonable part of a contour of an object can be determined based on the linked 
points. (Ramamoorthy,[0156], lines 1-3 where "a three-dimensional surface model 
estimate" which is derived from "transformed contours" is read as "a reasonable 
part of a contour of an object" that the set of N points [0150] from the 
"transformed contours" is read as the "linked points"). 

Ramamoorthy does not specifically disclose "project the reconstructed points into each 
image, and link, for each image, reconstructed points that are not projected at a junction 
between different contours or their projections to each other in order to provide a first 
set of links." 

Fitzgibbon, however, specifically discloses "project the reconstructed points into each 
image, and link, for each image, Fitzgibbon, section 3.2, paragraph 1, lines 3-5, 
where "3D point Xp" is read as the reconstructed point), reconstructed points that 
are not projected at a junction between different contours or their projections to each 
other in order to provide a first set of links, (Fitzgibbon, section 3.1, paragraph 1, 
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lines 8-10, where "3D point" is read as the reconstructed point, tracks are read as 
"linking", and the interface between the background and foreground objects are 
read as "a junction point between different contours")" 

It would have been obvious to one of ordinary skill in the art at the time of the invention 
to use the teachings of Fitzgibbon with the teachings of Ramamoorthy for improved 3-D 
object reconstruction because of the "advantages of multibody 3D reconstruction" 
(Fitzgibbon, Section 1, paragraph 6, line 1). 

9. As to claim 14 Ramamoorthy discloses, "Computer program product (44) for 
providing contour information related to images, comprising a computer readable 
medium having thereon: computer program code means, to make the computer, when 
said program is loaded in the computer: (Ramamoorthy, [0059], lines 1-5, where 
element 32 in figure 1 is read to a computer and computer code), obtain a set of 
interrelated images, (Ramamoorthy, [0017], lines 1-3, where "image data sets 
associated with an object" is read as "interrelated images"), segment said images, 
(Ramamoorthy, [0059], lines 23-26, where removal of the background is read as 
segmenting the images), extract at least two contours from the segmentation, 
(Ramamoorthy, [0129], lines 3-5 where "contours of each positive pixel group" is 
read as "at least two contours" further this process occurs in element 182 for 
figure 7 which is after the background removal, or segmentation in element 178), 
select interest points on at least some of the contours, (Ramamoorthy, [0133], lines 2- 
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4, where contour points are read as interest points), associate, for said extracted 
contours, interest points (J) with corresponding reconstructed points by means of three- 
dimensional reconstruction, (Ramamoorthy, [0156], lines 1-3, where transformed 
contours is read as extracted contours and the contour points are read as the 
interest points), such that at least a reasonable part of a contour of an object can be 
determined based on the linked points. (Ramamoorthy,[0156], lines 1-3 where "a 
three-dimensional surface model estimate" which is derived from "transformed 
contours" is read as "a reasonable part of a contour of an object" that the set of 
N points [0150] from the "transformed contours" is read as the "linked points"). 

Ramamoorthy does not specifically disclose "project the reconstructed points into each 
image, and link, for each image, reconstructed points that are not projected at a junction 
point between different contours to each other or their projections in order to provide a 
first set of links." 

Fitzgibbon however, specifically, discloses "project the reconstructed points into each 
image, (Fitzgibbon, section 3.2, paragraph 1, lines 3-5, where "3D point Xp" is 
read as the reconstructed point), and link, for each image, reconstructed points that 
are not projected at a junction point between different contours to each other or their 
projections in order to provide a first set of links," (Fitzgibbon, section 3.1, paragraph 
1, lines 8-10, where "3D point" is read as the reconstructed point, tracks are read 
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as "linking", and the interface between the background and foreground objects 
are read as "a junction point between different contours"). 

It would have been obvious to one of ordinary skill in the art at the time of the invention 
to use the teachings of Fitzgibbon with the teachings of Ramamoorthy for improved 3-D 
object reconstruction because of the "advantages of multibody 3D reconstruction" 
(Fitzgibbon, Section 1, paragraph 6, line 1). 

10. Claims 2-4, 7, 8, and 12 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over the combination of Ramamoorthy and Fitzgibbon, as applied to claim 
1 in view of Szeliski et al., ("Robust Shape Recovery from Occluding Contours Using a 
Linear Smoother", Journal International Journal of Computer Vision, Volume 28, 
Number 1 / June, 1998, Pages 27-44 herein after "Szeliski"). 

11. As to claim 2, the combination of Ramamoorthy and Fitzgibbon does not disclose 
that "wherein the step of linking in the first set of links comprises only providing links 
between reconstructed points or their projections associated with the same contour." 

However, Szeliski discloses "wherein the step of linking in the first set of links comprises 
only providing links between reconstructed points or their projections associated with 
the same contour." (Szeliski, section 2, paragraph 2, lines 1-4, where edgel is read 
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as a projection because it is a representation of the 3-D reconstructed point, 
section 2, paragraph 3, lines 6-9). 

It would have been obvious to one of ordinary skill in the art at the time of the invention 
to use the teachings of Szeliski with the teachings of the combination of Ramamoorthy 
and Fitzgibbon for improved 3-D object reconstruction because their "technique 
produces a continuous surface description, i.e., a network of linked 3D surface points, 
which provides us with a much richer description than just a set of 3D curves." 
(Szeliski, Section 1, paragraph 6, lines 1-4). 

12. As to claim 3, the combination of Ramamoorthy, Fitzgibbon and Szeliski disclose, 
"where the interest points comprise junction points (J), (Szeliski, section 2, paragraph 
2, lines 1-4, where edgels are read as interest points that comprise junction 
points since it is based on an edge from an edge detector, section 2, paragraph 1, 
lines 13-14, fig 1(b) where edge detector outputs are referred to as edgels.), where 
a junction point is provided at a location where two contours border each other." 
(Szeliski, section 2, paragraph 1, lines 4-6 , the examiner interprets edge points to 
be at a location where two contours border each other). 

13. As to claim 4, the combination of Ramamoorthy, Fitzgibbon and Szeliski disclose, 
"further comprising the step of combining, for a contour, the links in the first set of links 
provided in relation to each image for obtaining at least a reasonable part of a complete 
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contour of an object (step 40)." (Szeliski, section 2, paragraph 4, lines 7-8, where 
"virtual surface is read as "a reasonable part of a complete contour of an 
object"). 

14. As to claim 7, the combination of Ramamoorthy, Fitzgibbon and Szeliski disclose, 
"wherein the step of linking comprises linking, for each image, reconstructed points that 
are projected at a junction or their projections to reconstructed points or their projections 
in a second set of links." (Szeliski, section 5.3, paragraph 2, lines 1-7, where the 3D 
position of an edgels is read as a "reconstructed point" and the edgels represent 
edge points which are at "junctions", and the epipolar line defines the "second 
set of links" since the epipolar line contains all candidate edgels section 2, 
paragraph 3, lines 3-5. ). 

1 5. As to claim 8, the combination of Ramamoorthy, Fitzgibbon and Szeliski disclose, 
"wherein the reconstructed points that are projected at a junction in a majority of the 
images or their projections are linked in the first set of links." (Szeliski, section 5.3, 
paragraph 2, lines 1-7, where the 3D position of an edgels is read as a 
"reconstructed point" and the edgels represent edge points which are at 
"junctions", and the epipolar line defines the "first set of links" since the epipolar 
line contains all candidate edgels section 2, paragraph 3, lines 3-5) 
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16. As to claim 12, the combination of Ramamoorthy, Fitzgibbon and Szeliski teach 
the method according to claim 4, further comprising "the step of coding the images 
(Ramamoorthy, [0164], lines 15-16; where "image blocks" contain the images), 
(step 42), where the information about the linked reconstructed points is used in the 
coding. (Ramamoorthy, [0165], lines 7-9; where the encoded shape information 
results from the "three-dimensional surface model" [0156] that is made up of N 
contour points [0143] which are read as the "linked reconstructed points"). 

17. Claim 5 is rejected under 35 U.S.C. 103(a) as being unpatentable over the 
combination of Ramamoorthy, Fitzgibbon and Szeliski, as applied to claim 4 in view of 
Yim et al.,(US Patent Publication 20020136440 herein after "Yim"). 

18. As to claim 5, the combination of Ramamoorthy, Fitzgibbon and Szeliski do not 
specifically disclose, "wherein the step of combining comprises only combining the links 
to points that have less than three links." 

However, Yim specifically discloses, "wherein the step of combining comprises only 
combining the links to points that have less than three links." (Yim, [0075], lines 6-9, 
vertices are read as points and the lines connecting the vertices are read as links. 
The combining occurs with vertices on the same line which constrains the 
number of links between the points to less than three, which are then combined). 
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It would have been obvious to one of ordinary skill in the art at the time of the invention 
to use the teachings of Yim with the teachings of the combination of Ramamoorthy, 
Fitzgibbon, and Szeliski for improved 3-D object reconstruction because the invention 
enables "more accurate realistic CFD modeling of blood flow patterns." (Yim, [0021], 
lines 8-10). 

19. Claim 6 is rejected under 35 U.S.C. 103(a) as being unpatentable over the 
combination of Ramamoorthy, Fitzgibbon, Szeliski, and Yim, as applied to claim 5 in 
view of Ye Zhang et al., ("Robust 3D head tracking under partial occlusion", Proceedings 
of the Fourth IEEE International Conference on Automatic Face and Gesture 
Recognition, 2000, pages 176-182 herein after "Ye Zhang"). 

20. As to claim 6, the combination of Ramamoorthy, Fitzgibbon, Szeliski, and Yim 
does not specifically disclose, "further comprising the step of discarding, for each image, 
at least some of those reconstructed points or their projections to which links are 
provided from more than two other reconstructed points or their projections." 

Ye Zhang, however, specifically discloses, "further comprising the step of discarding, 
for each image, at least some of those reconstructed points or their projections to which 
links are provided from more than two other reconstructed points or their projections." 
(Ye Zhang, section 2.3, paragraph 5, lines 21-22, lines "V" is read as "at least 
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some of those projections", Section 2.2, lines 3-7, discarding and "F" is read as 
links ) 

It would have been obvious to one of ordinary skill in the art at the time of the invention 
to use the teachings of Ye Zhang with the teachings of the combination of 
Ramamoorthy, Fitzgibbon, Szeliski, and Yim for improved 3-D object reconstruction 
because their "new post-regularization method based on edge flow is designed to 
reduce the accumulation error." (Ye Zhang, Section 1.2, paragraph 3, lines 1-2). 

Conclusion 

21 . The prior art made of record and not relied upon is considered pertinent to 
applicant's disclosure. 

a. U.S. Patent No. 5933527 to Ishikawa, discloses contour extraction. 

b. U.S. Patent No. 6487304 to Szeliski, discloses 3D reconstruction from 2D 
images. 

c. U.S. Patent No. 6856314 to Ng, discloses 3D reconstruction from 2D 
images. 

d. Boyer et al., "Object models from contour sequences", Lecture Notes in 
Computer Science, 1996 - Springer, discloses building 3D object models from 
object contours. 
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e. Bascle et al., "Region tracking through image sequences", Computer 
Vision, 1995. Proceedings., Fifth International Conference on, pages 302-307, 
discloses contour based tracking with links. 

f. Lui et al., "Layered Representation of Scenes Based on Multiview Image 
Analysis", Circuits and Systems for Video Technology, IEEE Transactions on, 
June 2000, Volume 10, Issue 4, pages 518-529, discloses contour tracking with 
3D reconstruction. 

g. Heigl et al., "An Efficient Combination of 2D and 3D Shape Descriptions 
for Contour Based Tracking of Moving Objects," Book Series Lecture Notes in 
Computer Science, Publisher Springer Berlin / Heidelberg, Volume 1406/1998, 
discloses 2D contour tracking with 3D reconstruction 

h. Mech et al., "A noise robust method for 2D shape estimation of moving 
objects in video sequences considering a moving camera", Signal Processing, 
1998, pages 203-217, discloses shape tracking with 3D reconstruction 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to PAUL OTTO, JR whose telephone number is (571)270- 
3391 . The examiner can normally be reached on Monday - Thursday 7:30am-5:00pm. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Nabil El-Hady can be reached on (571) 272-3963. The fax phone number 
for the organization where this application or proceeding is assigned is 571-273-8300. 
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Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a 
USPTO Customer Service Representative or access to the automated information 
system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 
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Examiner, Art Unit 4146 
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